Linear programming-based solution methods for constrained partially observable Markov decision processes

Constrained partially observable Markov decision processes (CPOMDPs) have been used to model various real-world phenomena. However, they are notoriously difficult solve optimality, and there exist only a few approximation methods for obtaining high-quality solutions. In this study, grid-based approximations in combination with linear programming (LP) models generate approximate policies CPOMDPs. A detailed numerical study is conducted six CPOMDP problem instances considering both their finite infinite horizon formulations. The quality of algorithms solving unconstrained POMDP problems established through comparative analysis exact solution methods. Then, the performance LP-based approaches varying budget levels evaluated. Finally, flexibility demonstrated by applying deterministic policy constraints, investigation into impact on rewards CPU run time provided. For most problems, constraints found little expected reward, but introduce significant increase time. reverse observed: tend yield lower total than stochastic counterparts, negligible case. Overall, these results demonstrate that LP can effectively while providing incorporate additional underlying model.